Limited bandwidth to affect processor design

T he phenomenal improvements in microprocessor performance place significant demands on memory systems , requiring a low latency, high-band-width stream of operands. Researchers point out that DRAM access latencies (measured in processor cycles) are growing and any request that misses in the caches may eventually take hundreds of cycles to satisfy. These researchers have proposed many techniques to mitigate the penalties of long memory latencies, such as lockup-free caches, cache-conscious load scheduling, hardware and software prefetching, stream buffers, speculative loads and execution, multithreading, data value prediction, and instruction reuse. Most of these techniques, while reducing the impact of contentionless access laten-cies, do so at the cost of increasing a pro-gram's bandwidth requirements. These latency tolerance (or reduction) techniques may increase a processor's memory band-width needs by causing the processor to request the same stream of operands in less time, or by causing the processor to request more data from memory. In turn, these techniques may cause the processor to stall due to queueing in the memory system. (We differentiate between " intrinsic latency " of a contentionless access and latency added by queueing in the memory system, which results from limited bandwidth.) Neither the long latencies nor the increased bandwidth requirements constitute a " memory wall " that will eventually inhibit improved microprocessor performance. Instead, designers will employ a range of design decisions and new technologies to produce balanced, cost-effective systems. The extent to which long latency and bandwidth requirements affect performance will determine which techniques or technologies are affordable, and/or worth the effort of implementing. Since some of the solutions trade improved latency for increased traffic, or higher bandwidth for increased latency, the relative effects that these two components of memory accesses have on performance is important. Here, we quantify and compare the performance impacts of memory latencies and finite bandwidth. We show that the implementation of aggressive latency tolerance techniques aggravates stalls due to finite memory bandwidth, which actually become more significant than stalls resulting from uncongested memory latency alone. We expect that memory bandwidth limitations across the processor pins will drive significant architectural change, for the following reasons: • Continuing progress in processor design will increase the issue rate of instructions. These advances include both architectural innovation (wider issue, speculative execution, and so forth) and circuit advances (faster, denser logic). • To the extent that latency tolerance techniques are successful, they will speed up the retirement rate …